…ntation

This commit introduces a comprehensive design and implementation plan for a new ClickHouse log sink within the Ion observability library. The document outlines the context, architecture, package structure, and component specifications for integrating ClickHouse as a logging sink, enabling efficient analytical queries over log data. The new package will be built independently and will not affect existing functionality until validated.

This commit adds several indirect dependencies to the Go module, including updates for ClickHouse libraries, Brotli compression, and various utility packages. The changes enhance the project's capabilities and ensure compatibility with the latest versions of these libraries.

This commit introduces a new configuration package for the ClickHouse log sink, including a Config struct that holds connection details, default values, and validation methods. The package also includes a Ping function to check the connectivity to ClickHouse. Comprehensive tests have been added to ensure the correctness of configuration validation and default value application.

…ouse

This commit introduces a new schema management system for ClickHouse, including the implementation of DDL generation and schema validation. The `buildDDL` function constructs the SQL create table statement based on a provided table name, while the `EnsureSchema` function ensures the schema is created if it does not exist. Additionally, comprehensive unit tests have been added to validate the DDL generation and schema enforcement logic.

This commit introduces a new batch writing mechanism for ClickHouse logs, allowing for efficient handling of log rows. The `batchWriter` struct manages a buffered channel for log rows, with configurable batch size and flush intervals. It includes methods for sending log rows, flushing batches, and tracking dropped rows. Additionally, unit tests have been added to ensure the functionality of the batch writing process.

…lity

This commit introduces a comprehensive suite of unit tests for the `batchWriter` and `Core` components of the ClickHouse logging implementation. The tests cover various scenarios including batch flushing, error handling, and synchronization, ensuring robust functionality. Additionally, the `Core` tests validate the zapcore interface and configuration handling, enhancing overall reliability and correctness of the logging system.

…ation

This commit updates the `logRow` struct to include ClickHouse-specific tags for each field, improving schema mapping. Additionally, it introduces a regex validation for table identifiers in the configuration, ensuring that invalid table names trigger a panic during initialization. The validation logic is expanded to include checks for various configuration parameters, enforcing stricter rules on their values. New error messages are added to provide clearer feedback on configuration issues.

This commit adds new unit tests for the `extractRow` function, covering scenarios for large and small uint64 values, time fields, and complex numbers, ensuring correct handling and storage in the appropriate fields. Additionally, it introduces a new configuration builder with setter methods for various parameters, improving the configuration management process. The validation tests have been updated to ensure stricter checks on configuration values, enhancing overall robustness.

This commit updates the ClickHouse configuration to include an `Enabled` field, allowing for conditional activation of the log sink. It also improves validation logic to ensure that the DSN is provided when the ClickHouse sink is enabled. Additionally, new unit tests are added for the `extractRow` function to handle byte strings and stringer types, ensuring accurate logging of these fields. The batch writing tests have been refactored for clarity and consistency.

This commit introduces the ClickHouse logging functionality into the Ion framework. It adds a new `chCore` field to the `Ion` struct for managing ClickHouse interactions and updates the logger setup to include ClickHouse-specific logging capabilities. A new `DroppedCount` method is implemented to track log entries dropped by the ClickHouse sink. Additionally, integration tests are added to validate the logging behavior and ensure correct data handling in ClickHouse. Configuration validation is enhanced to support ClickHouse logging levels.

This commit refactors the ClickHouse logging implementation by moving the ClickHouse configuration package to an internal directory. It updates import paths accordingly to reflect the new structure. Additionally, new test files for batch writing and core functionality are introduced, enhancing the test coverage for the ClickHouse integration. This change aims to improve the organization and maintainability of the codebase.

…ouse configuration

This commit removes the deprecated 'warning' log level from the valid levels in the ClickHouse configuration. Additionally, it introduces default values for the ClickHouse configuration parameters, ensuring a more robust setup for the logging functionality. The changes aim to streamline the configuration process and improve overall clarity in the logging levels supported.

Add/clickhouse

…tion

This commit updates the README.md to include comprehensive documentation for the ClickHouse configuration options. It details the parameters available for the ClickHouse log sink, including their types, defaults, and descriptions. Additionally, it provides the SQL schema for the ClickHouse table and guidelines for monitoring back-pressure, enhancing the clarity and usability of the documentation for users integrating ClickHouse logging.

docs(readme): add ClickHouse configuration details and schema informa…